1. Field of the Invention
The present invention is a method and system to detect and track shopping carts from video images in a retail environment.
2. Background of the Invention
Video analytics in the field of retail marketing provides a wealth of information from video images collected from cameras placed in stores. The analysis mostly concerns how shoppers navigate the stores and make purchase decisions. In-store shopper behavior can be summarized in one way by their tracks in stores—which aisles they visit and how much time they spend in different product categories. Current state-of-the-art video processing technology can track customers throughout the store and generate tracking data.
In many of the stores, shopping is aided by shopping carts. From the viewpoint of video analysis technology, the presence of shopping carts can be a nuisance; shopping carts often confuse an automatic tracking algorithm, causing it to detect and track carts as shoppers. On the other hand, the shopping carts may provide additional valuable marketing information, because the trajectory of a cart effectively summarizes the shopping trip of a person or a group. Especially when shoppers come in groups, the track of a shopping cart may reveal very informative clues about the shopping trip, while tracks of some group members (such as children) can be less significant.
Many video-based surveillance and monitoring technologies utilize motion information to find and track objects of interest—such as humans, vehicles, etc. Motion-based object detection/tracking has been successfully and extensively used due to its simplicity and efficiency. For in-store video analytics, the presence of shopping carts poses challenges under this framework; because shopping carts move in the same manner as the shoppers who are pushing the carts, a shopper using a cart may be confused as a group of people. Such errors will corrupt the automatically-generated shopping behavior data.
The present invention addresses such issue in automatic retail video analytics—the problem of differentiating carts from groups of people—so that carts can be detected and tracked. The present invention takes advantage of the distinct motion features of a person pushing a cart that are different from the motions within a group of people. The motion of a cart is initiated by the person pushing (or pulling, in some cases) it; because a cart is a passive object, it moves only whenever a person moves it. The first step is to find any regions in a video frame that are moving. The object of interest—a cart—always belongs to a shopper when it is in motion, and the person pushing a cart often generates a single motion blob larger than a person. In a retail environment, a large motion blob detected from a view of the store floor contains either a shopper (or shoppers) with a cart or just multiple shoppers without carts. Whenever a motion blob is found that is of a size large enough to contain a person with a cart, the system examines whether or not the blob contains a cart. First, linear motion edges are found and tracked; the linear motion edges are the characteristic image features of a moving cart. If the positions and motions of these linear edges match the characteristic motion of a cart, then the motion blob is further considered as a serious candidate for a “cart candidate blob”—a blob containing a person (or more than one person) and a cart. Further steps of extracting relative motion features within the candidate blobs and comparing the extracted motion features to a model of characteristic motions of a cart blob finally determine whether or not the candidate blob indeed contains a cart. The system keeps updated positions of the detected carts so that they are individually tracked.
There have been prior attempts for tracking the motion of carts or baskets for the purpose of understanding shoppers' behaviors.
U.S. Pat. No. 6,659,344 of Otto, et al. (hereinafter Otto) presents a shopper behavior monitoring system using RFID tags attached to products and RFID scanners installed in shopping baskets, so that the system can detect product purchase at the shelf and identify the purchase items. In U.S. Pat. Appl. Pub. No. 2008/0042836 of Christopher (hereinafter Christopher), the RFID system is used to track shopping carts throughout the store. The present invention utilizes video cameras to detect carts and generate their trajectories without using any costly and cumbersome devices. The motion blob detection and tracking utilizes a method similar to “Measuring concavity on a rectangular mosaic”, IEEE Transaction on Computers, Volume 21, by Sklansky (hereinafter Sklansky).
In summary, the present invention provides an approach to detect and track carts in retail environments. Unlike some of the prior inventions, the present invention does not require any specialized device to detect and track shopping carts. The present invention takes advantage of the rigid motion signature of a shopping cart in motion, and also the relation between the motion of the cart and the motion of the shopper pushing the cart.